Tricks of the Mac Game Programming Gurus/TricksOfTheMacGameProgrammingGurus.iso/More Source/Libraries/VideoToolbox 95.04.18/Notes/cscSetModeTiming.note

Internet Message Format | 1995-04-13 | 18.4 KB | [TEXT/R*ch]

From: Denis Pelli (3/17/1995)
To:   Apple Developer Support,   Apple PCI Bus Development
CC: David Nagel
Subject:Video Drivers            5:06 PM        3/17/1995
March 17, 1995

Dear Apple,

We, the 41 undersigned visual scientists [and another ten who endorsed the
letter after it was sent], are writing to suggest the addition of a new
"cscSetModeTiming" video driver control call to the "Display Driver Calls"
chapter of "Designing PCI Cards & Drivers".

THE NEED:

Today's video cards and monitors are mostly capable of running with a
practically infinite range of resolutions and field rates (timings).
Current MacOS software, recently enhanced by the addition of the Display
Manager, restricts this infinite spectrum to a short list of modes
(timings) specified in the video card's ROM and driver. This creates a
perennial compatibility problem as old video cards cannot drive new
displays with unanticipated timings. (In principle a new driver could be
installed, but this would require writing a new driver for each video
card that is to support the new display, which is impractical.) New
display devices--LCD, projection, HDTV, stereo, etc.--are coming and it
seems likely that they will function best with new timings that are hard
to anticipate today.

One of the best ways of generating stereo imagery (for virtual reality
and cyberspace) is to double the monitor's frame rate (e.g. to 134 Hz)
and to use LCD glasses that allow each eye to see alternate frames. Many
commercially available video monitors will run at such high frame rates,
but, alas, most Mac video drivers have no provision for it, even though
the video card is capable.

The availability of custom timings would provide a new opportunity
for game developers. Games often achieve exciting effects by pushing the
hardware to its limits. Some very interesting effects could be generated
by smoothly changing the timing from spatially coarse high frame rate
images for fast motion sequences to spatially fine temporally slow (e.g.
67 Hz) frame rates for intricate imagery.

Many vision studies require high frame rates, over 100 Hz, in order to
study stereo, motion perception, and retinal phenomena without
artifacts. Many monitors and most video cards would run happily at such
high frame rates, but hardly any of today's Mac video drivers will allow
it.

TESTIMONIALS TO THE NEED:

Patrick Cavanagh, [e-mail], writes "High-frame-rate
video cards are available for the PC, typically for stereo. None are
available for the Mac. Many of us and those in computer graphics are
being forced to the PC market because of this oversight."

James Dinunzio, [e-mail], writes, "Hi. I am a graduate
student at University of Southern Cal in the Computer Science
Department. I am a member of a newly formed group working under faculty
dedicated to graphics and virtual reality. I am currently working on a
project to produce decent stereopsis on the Macintosh, including still
stereo images, movies, and if possible "live" walkthroughs. I will be
using a pair of LCD shutter glasses controlled through the serial port
and running at a nominal 30 fps, 60fps for both eyes. ... Any
suggestions?"

Tom Busey, [e-mail], adds, "I have just finished writing code
that displays color stereo 3-d pairs for use with LCD glasses. This code
uses sequential-frame presentations synched with LCD glasses controlled
via the serial port to provide the illusion of depth. While this
technology is old, what is new is my code that takes two pict files and
reduces the colors from millions of colors to what amounts to 4 bits per
red, green and blue channel for each picture (on monitors with millions
of colors) or 2 bits per channel for monitors with thousands of colors.
The two pictures are then added together and then I use clut switching
to switch between the two pictures. The stereo 3-d code uses calls from
(not surprisingly) the VideoToolbox code library [described below]. As
with all the code I write, this code is freely available to anyone who
wants it. I'm currently working with LCD glasses distributors to have it
distributed free to purchasers of LCD glasses. It may be downloaded
from:"
ftp://ftp.psych.indiana.edu/pub/busey/AdjustStereoPicts.sea.hqx

"The bottom line is that this approach makes virtual reality a (virtual)
reality on macs, for very low cost (about $150 for the glasses, plus
some parts for a very simple interface). However, because of the
relatively slow frame rate of my 2-page monitor (67 hz), I am only
getting about 33 images per eye per second. This causes visible flicker,
and I would love to be able to increase the frame rate to 120 hz. Such
an increase would not only reduce the flicker, but it would also make
the glasses and the mac research-grade for doing experiments. Thus I
support your call for Apple to add an adjustable frame rate option."

[The following testimonials were received after the original
letter was sent. 3/22/95]

Mats Lind,  [e-mail], writes, "3-D Max, a Swedish company,
working with a Korean supplier, has just started to deliver attractively
priced LCD-shutter glasses and accessories giving binocular stereo
capabilities to PCs. Their product received enormous attention at the
Ce-Bit fair in Hannover last week. The estimated retail price in the US
will be less than 180 USD (including some extra software like 3-D games)
and will probably reach a high market share. 3-D Max, is very interested
in entering the Apple market segment as well, but to do so requires the
usual 120+ Hz."

Chris Tyler, [e-mail], writes, "We are using the VideoToolbox
to develop a two-monitor stereo display system. We would be delighted
to know of calls that could control the raster rates and video card
synchronization. I think that I can speak for many investigators at the
Smith-Kettlewell Institute, which probably owns about 50 Macintoshes,
that the features you propose would be of great value to many."

FIVE SUGGESTIONS TO APPLE:

We suggest the following additions to the "Display Driver Calls" chapter
of "Designing PCI Cards & Drivers".

1. (MAJOR) Add a cscSetModeTiming control call to the repertoire for
video drivers to allow specification of an arbitrary new timing.

Using an extended VDTimingInfoRec struct, or perhaps an entirely new
struct, the calling program would specify the desired timing (i.e. pixel
rate, line rate, field rate, frame synch width, frame front porch, frame
back porch, frame synch polarity, line synch width, line front porch,
line back porch, line synch polarity, synch on green (y/n), combined
synch polarity, and interlaced/noninterlaced, as well as the desired
rowBytes and page [oops, I meant "pageBytes"]) and issue a
cscSetModeTiming control call to the video driver. In response, the
video driver would change the mode to comply, as nearly as possible,
with the request, and, before returning, would update the caller's
struct to reflect the achievable timing. The driver would also be
responsible for updating the mode's other derivative specifications
(bounds rect, etc.) that are used by other control and status calls
(e.g. cscSwitchMode, cscGetPages, cscGetModeTiming,
cscGetNextResolution, cscGetVideoParams). The effect of this change
should be persistent, lasting at least until shut down.

(Timing changes that do not affect rowBytes would not require calling
the Display Manager to repaint the screen, so most timing transitions
could be done smoothly, without any visible glitches. The application
would typically call DMSetDisplayMode only at the end of the special
effect, mainly to update the bounds rect.)

(cscSetModeTiming is a powerful call that could create timings that
would not effectively drive the monitor, and might even harm it.
However, this power can be accessed only through a Control call to the
driver, and thus is inaccessible to the casual user. We would expect it
to be used only by applications written by the monitor manufacturers and
developers of games and virtual reality environments and other
special-effects applications.)

cscSetModeTiming would allow an application run at start up to customize
ANY video card (whose driver supports the call) to drive a novel
display, or drive an old display at novel timings. The new timing would
persist until shut down or restart.

2. (MINOR) It would be helpful to enhance the corresponding
cscGetModeTiming status call to return the same timing info.

3. (MINOR) Please add a trivial cscGetVBLBitAddress status call to
return the location of the VBL bit. As far as we know, every Mac video
device has a VBL bit, somewhere, that is true only during blanking.
Unfortunately there is no easy way to find it. For many special effects
it is necessary to synchronize multiple video monitors. Since every
video card insists on being a master, and they have independent clocks,
monitors normally run asynchronously of one another. In the past we've
used custom hardware and software tricks, specific to particular video
cards, to reset multiple cards at once, or to halt the clock on one
until it drifts into phase with another. A better, general, solution
would be to use cscSetModeTiming to tweak the timing of one video card
relative to the other until they come into synch. However, to do this,
we need to be able to continuously monitor the VBL bit of each device.
(Using a VBL interrupt service routine to set a bit at a known location
is much less satisfactory because of the extra uncertainty in timing and
because this gives us only the time of onset of VBL, denying us access
to the time of offset.)

4. (VERY MINOR) We'd like to request that a clarification be added to
the documentation of the cscSetGray control call. Most video drivers
implement this call correctly. However, a few video drivers (e.g.
PowerMac 7100 vram driving an Apple 2-page monochrome monitor, and the
Radius PrecisionColor 8 and 8Xj driving a Radius 2-page monochrome
monitor), upon detecting the presence of a monochrome video monitor,
lock themselves into the monochrome state (i.e. always do luminance
mapping) irrespective of calls to cscSetGray. The cscSetGray calls
return without error and calls to cscGetGray misleadingly imply that the
mono/color mode is being changed, but the driver stubbornly continues to
do luminance mapping. This prevents us from using some very useful
special effects that depend on having full control of the CLUT (i.e.
color mode) while driving a monochrome display (see Pelli & Zhang,
1991). Please tell the developers of video drivers not to get tricky:
don't disable cscSetGray.

5. (VERY MINOR) A surprisingly large number of video drivers (e.g. Mac
IIci Built-In Video, Relax 19" Model 200) fail to correctly implement
cscGetEntries, even though it is a required status call. Perhaps a
sentence could be added to the documentation of cscGetEntries, advising
developers to perform a write-then-read (cscSetEntries & cscGetEntries)
test of their new driver before releasing it to the world. (TimeVideo,
described below, does this test, which revealed the bugs.)

Denis Pelli
Apple Partner
Professor of Neuroscience at Syracuse University
[e-mail]

Kenneth R. Alexander
Eye and Ear Infirmary
University  of Illinois
[e-mail]

Michael Bach
Dept. of Ophthalmology
University of Freiburg
Germany
[e-mail]

Martin S. Banks
School of Optometry
UC Berkeley
[e-mail]

Paul Beckmann
Psychology Department
University of Minnesota
[e-mail]

Patrick Bennett
Psychology Department
University of Toronto
[e-mail]

Margaret Bradley
NIMH Center for the Study of Emotion and Attention
University of Florida.
[e-mail]

David Brainard
Psychology Department
UC Santa Barbara
[e-mail]

Catherine Burns
Institute for Sensory Research
Syracuse University
[e-mail]

Tom Busey
Department of Psychology
Indiana University
[e-mail]

Patrick Cavanagh
Psychology Department
Harvard University
[e-mail]

Frans W. Cornelissen
Laboratory for Experimental Ophthalmology
Center for Behavioural, Cognitive and Neurosciences
University of Groningen
The Netherlands
[e-mail]

Andrew Derrington
Psychology Dept
Nottingham University
UK
[e-mail]

Michael P. Eckert
School of Electrical Engineering
University of Technology, Sydney
Australia
[e-mail]

Rhea T. Eskew, Jr.
Dept. of Psychology
Northeastern University
[e-mail]

Bart Farell
Institute for Sensory Research
Syracuse University
[e-mail]

Mark W. Greenlee
Neurologische Universitaetsklinik
University of Freiburg
GERMANY
[e-mail]

Stevan Harnad
Department of Psychology
University of Southampton
UK
[e-mail]

Mary Hayhoe
Center for Visual Science
University of Rochester
[e-mail]

Alun Johns
Psychology Department
Oxford University
UK
[e-mail]

Keith Karn
Center for Visual Science
University of Rochester
[e-mail]

Stanley Klein
School of Optometry
UC Berkeley
[e-mail]

Peter Lennie
Center for Visual Science
University of Rochester
[e-mail]

Dennis Levi
School of Optometry
University of Houston
[e-mail]

Donald I. A. MacLeod
Psychology Department
University of California S.D.
[e-mail]

Tony Norcia
Smith Kettlewell Eye Research Institute
San Francisco, CA
[e-mail]

Joel Pokorny
Eye Research Lab.
University of Chicago
[e-mail]

Bertram O. Ploog
Hunter College
CUNY
[e-mail]

Russell A. Poldrack
Amnesia Research Lab
Beckman Institute
[e-mail]

Manoj Raghavan
Institute for Sensory Research
Syracuse University
[e-mail]

Allison B. Sekuler
Psychology Department
University of Toronto
[e-mail]

Shui-I Shih
Psychology
Southampton University
UK
[e-mail]

Ben Singer
Center for Visual Science
U Rochester
[e-mail]

Vivianne C. Smith
Eye Research Lab.
University of Chicago
[e-mail]

Joshua Solomon
Vision Group
NASA Ames
[e-mail]

Hans Strasburger
Institut fuer Medizinische Psychologie
University of Muenchen
Germany
[e-mail]

Joann M. Taylor
Color Technology Solutions
[e-mail]

Marty Wachter
School of Medicine
Johns Hopkins University
[e-mail]

Andrew B. Watson
Vision Group
NASA Ames
[e-mail]

Lan Zhang
Psychology Dept.
Syracuse University
[e-mail]

Yi-Xiong Zhou
Center for Neural Systems
NYU
[e-mail]

[In addition to the above 41, the following 10 people also asked
to be included as signatories, but I received their replies too
late for the original letter. 3/27/95]

Lawrence K. Cormack
Psychology Department
University of Texas at Austin
[e-mail]

James Dinunzio
Computer Science Department
USC
[e-mail]

Lewis O. Harvey Jr.
Department of Psychology
University of Colorado
[e-mail]

Ju Li
Department of Psychology
Columbia University
[e-mail]

Mats Lind
Center for Human Computer Studies
Uppsala University
Sweden
[e-mail]

Walter Makous
Center for Visual Science
University of Rochester
[e-mail]

David Rose
Department of Psychology
University of Surrey
UK
[e-mail]

Stefan Treue
Div. of Neuroscience
Baylor College of Medicine
[e-mail]

Christopher Tyler
Smith Kettlewell Eye Research Institute
San Francisco, CA
[e-mail]

Hugh R. Wilson
Visual Science Center
University of Chicago
[e-mail]

BACKGROUND--THE VIDEOTOOLBOX:

Denis Pelli uses Macs to do vision research. A '91 paper of his
described a simple electronic circuit that combined the three DAC
outputs of a color video card to achieve exquisitely precise grayscale
images on a grayscale monitor. A recent paper, on how people see
letters, was the cover article of the June '94 issue of Nature.
(Reprints are available.) A paper on how we see words is currently under
review at Science. Over the years since the Mac II appeared he developed
a library of C software (the VideoToolbox) for doing vision research on
Macs, and since '91 the complete archive has been available to the
public. It is electronically available in source form from info-mac,
updated several times a year, and is free for research purposes. We
estimate that one hundred vision research labs worldwide are using this
software, each with many Macs and students.
ftp://sumex-aim.stanford.edu/info-mac/dev/src/video-toolbox-95-01-14-c.sit

The TimeVideo application included in the VideoToolbox checks out the
timing of all video devices in anticipation of their use in critical
real-time applications, e.g. movies or lookup table animation. It seems
to be the only test program in existence for rigorous testing of Mac
video drivers. It uncovered bugs in drivers from Dome, Radius,
RasterOps, Relax, SuperMac, TrueVision, and Apple, including the Mac
IIci "Macintosh II Built-In Video" (.Display_Video_Apple_RBV1), 4*8 and
8*24 "Macintosh Display Card" (.Display_Video_Apple_MDC), "Macintosh
Display Card 8*24 GC" (.Display_Video_Apple_MDCGC), and Quadra 700, 900,
and 950 (.Display_Video_Apple_DAFB versions 0 and 2). Version
3 of the .Display_Video_Apple_DAFB driver (e.g. in the Centris 650) is
bug-free. All bugs have been reported to the manufacturers. We're happy
to report that all of Apple's new drivers published in the last several
years are bug free. Perhaps the driver-design teams have added TimeVideo
to their testing suite.
ftp://sumex-aim.stanford.edu/info-mac/Configuration/time-video-363.sit

Pelli, D. G. and Zhang, L. (1991) Accurate control of contrast on
microcomputer displays. Vision Research, 31, 1337-1350.

Solomon, J. A. and Pelli, D. G. (1994) The visual filter mediating
letter identification. Nature, 369 (6479), 395-397.

Pelli, D. G., Farell, B., and Moore, D. C. (1995) The role of letters in
recognizing words. Under review at Science.

Burns, C. W., Pelli, D. G., Farell, B., and Moore, D. C. (1995)
Identifying letters. Submitted to Vision Research.

TYPOS:

In the "Display Driver Calls" chapter of interim-release "Designing PCI Cards &
 Drivers" on the MacOS SDK #2.
p. 161 "Too improve ..."
p. 163 "... 32-bit address#."
p. 163 "lumimance"
p. 163 "luminnace-"
p. 167 "The Display Manager. also ..."

Incidentally, the "Display Driver Calls" chapter would be easier to use
if it were a complete document, not requiring reference to the old 68k
"Designing Cards & Drivers".

ACKNOWLEDGEMENTS:

Thanks to Mike Landy for help with the timing specs.

======================================================================